Skip to content

fix: orchestrator enqueue yield#21286

Merged
PhilWindle merged 1 commit intomerge-train/spartanfrom
ag/refactor-orch-workers
Mar 13, 2026
Merged

fix: orchestrator enqueue yield#21286
PhilWindle merged 1 commit intomerge-train/spartanfrom
ag/refactor-orch-workers

Conversation

@alexghr
Copy link
Contributor

@alexghr alexghr commented Mar 10, 2026

Fixes the orchestrator limiting the number of in-progress jobs, now it only limits the actual enqueuing operation and gives the event loop more room to breath.

Either this PR must be backported or the previous PR #21247 must be removed the from the backport train.

// Workers yield between jobs via await, allowing I/O callbacks to process.
void this.deferredJobQueue.put(() => safeJob());
void this.deferredJobQueue.put(async () => {
void safeJob();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can also do await safeJob() but I guess you'd have to fight the linter. (any await yields)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awaiting here would introduce a limit on how many proofs we have in flight at a time. We want to use fire-and-forget here and use callbacks to continue up the rollup

void this.deferredJobQueue.put(() => safeJob());
void this.deferredJobQueue.put(async () => {
void safeJob();
// we yield here to the macro task queue such to give Nodejs a chance to run other operatoins in between enqueues
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really understand this change. Doesn't this negate the purpose of having a queue and effectively take us back to setImmediate(() => void safeJob())?

@PhilWindle PhilWindle merged commit 5508648 into merge-train/spartan Mar 13, 2026
16 of 20 checks passed
@PhilWindle PhilWindle deleted the ag/refactor-orch-workers branch March 13, 2026 09:43
AztecBot pushed a commit that referenced this pull request Mar 13, 2026
Fixes the orchestrator limiting the number of in-progress jobs, now it only limits the actual enqueuing operation and gives the event loop more room to breath.

Either this PR must be backported or the previous PR #21247 must be removed the from the backport train.
@AztecBot
Copy link
Collaborator

✅ Successfully backported to backport-to-v4-staging #21477.

AztecBot added a commit that referenced this pull request Mar 14, 2026
BEGIN_COMMIT_OVERRIDE
fix: orchestrator enqueue yield (#21286)
chore: default multiplier (#21482)
chore: update yarn.lock (#21479)
chore: backport #21231 feat!: improve L2ToL1MessageWitness API to v4
(#21463)
feat(rpc): add package version to RPC response headers (#21526)
feat(ethereum): check VK tree root and protocol contracts hash in rollup
compatibility (#21537)
feat: add public log filtering by tag (#21561)
END_COMMIT_OVERRIDE
github-merge-queue bot pushed a commit that referenced this pull request Mar 16, 2026
BEGIN_COMMIT_OVERRIDE
feat: add ETHEREUM_HTTP_TIMEOUT_MS env var for viem HTTP transport
(#20919)
fix(archiver): filter tagged log queries by block number (#21388)
fix(node): handle slot zero in getL2ToL1Messages (#21386)
feat(sequencer): redistribute checkpoint budget evenly across remaining
blocks (#21378)
fix: fall back to package.json for CLI version detection (#21382)
chore: Removed multiplier config (#21412)
chore: Removed default snapshot url config (#21413)
chore: Read tx filestores from network config (#21416)
fix(node): check world state against requested block hash (#21385)
feat(p2p): use l2 priority fee only for tx priority (#21420)
feat(p2p): reject and evict txs with insufficient max fee per gas
(#21281)
revert "feat(p2p): reject and evict txs with insufficient max fee per
gas (#21281)" (#21432)
chore: Reduce log spam (#21436)
fix(tx): reject txs with invalid setup when unprotecting (#21224)
fix: orchestrator enqueue yield (#21286)
chore(builder): check archive tree next leaf index during block building
(#21457)
fix: scenario deployment (#21428)
chore: add claude skill to read network-logs (#21495)
chore: update claude network-logs skill (#21523)
feat(rpc): add package version to RPC response headers (#21526)
chore(prover): silence "epoch to prove" debug logs (#21527)
chore(sequencer): do not log blob data (#21530)
fix: dependabot alerts (#21531)
docs(p2p): nicer READMEs (#21456)
fix(archiver): guard getL1ToL2Messages against incomplete message sync
(#21494)
fix(sequencer): await syncing proposed block to archiver (#21554)
feat(ethereum): check VK tree root and protocol contracts hash in rollup
compatibility (#21537)
fix: marking peer as dumb on failed responses (#21316)
fix(kv-store): make LMDB clear and drop operations atomic across
sub-databases (#21539)
feat(world-state): add blockHash verification to syncImmediate (#21556)
chore(monitor): print out l2 fees components (#21559)
chore: rm faucet (#21538)
chore: remove old merkle trees (#21577)
feat: Implement commit all and revert all for world state checkpoints
(#21532)
chore: skip flaky browser acir tests in CI (#21596)
fix: Better detection for epoch prune (#21478)
chore: logging (#21604)
fix: Don't update state if we failed to execute sufficient transactions
(#21443)
END_COMMIT_OVERRIDE
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants